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Modification  of  Current  Feedback  Strategies: 

A  Text  Synthesis  Approach 
Abstract 

Two  passages,  each  consisting  of  a  13  and  19  sentence  version,  were 
constructed  from  the  same  essay  on  the  development  of  the  atom  bomb.  The 
passage  sentences  were  individually  typed  on  index  cards,  and  the  four  resulting 
packets  were  then  scrambled.  Subjects  were  undergraduate  psychology  students  at 
the  University  of  Colorado  who  received  one  of  the  four  scrambled  orders  of 
sentences.  Subjects  serially  reconstructed  the  passages  using  a  slotted  board. 

In  the  feedback  condition  subjects  were  given  five  tokens  to  determine  if  a 
card  had  been  appropriately  placed.  The  no-feedback  students  did  not  receive 
any  assistance  in  determining  the  correctness  of  placement.  In  both  conditions 
subjects  signalled  when  they  thought  they  were  through. 

The  dependent  measures  were  time  (in  minutes),  number  of  moves,  concordance 
with  the  author's  original  order  of  sentences,  percentage  recall  (the  number  of 
idea  units  recalled),  and  percentage  recognition  (choosing  the  original  sentence 
from  a  pair  containing  a  paraphrase).  The  recall  and  recognition  measures  were 
given  directly  after  completion  of  the  task.  The  independent  variables  were 
feedback/no- feedback,  content,  and  number  of  sentences. 

MANOVA  and  MANCOVA  analyses  suggested  that  content  and  number  of  sentences 
exerted  a  greater  influence  on  processing  and  subsequent  recall  and  recognition 
than  did  the  presence  or  absence  of  feedback.  Concordance  with  the  original 
order  of  sentences  was  not  critical  in  determining  subsequent  recall  or 
recognition.  Protocol  analyses  confirmed  this  finding  regarding  concordance. 

With  content  and  passage  length  as  random  rather  than  fixed-effects,  the 
statistical  significance  of  these  findings  suggest  that  much  of  the  previous 
feedback  literature  may  have  been  experimentally  confounded.  In  addition,  the 
text  synthesis  approach  more  clearly  demonstrates  the  idiosyncratic  nature  of 


cognitive  processing  and  the  difficulty  of  providing  valid  feedback  for  the 
Individual  student.  It  was  proposed  that  the  text  synthesis  approach  be  used  to 
direct  feedback  towards  a  free  choice/menu  model.  Further  extensive 
experimental  work  along  these  lines  Is  needed. 
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Modification  of  Current  Feedback  Strategies:  A  Text  Synthesis  Approach 

Educational  specialists  are  currently  under  pressure  to  demonstrate  an 
Increased  response  to  demands  for  individualized  instruction.  The  stress  on 
individualization  reflects  increased  emphasis  on  mastery  learning  (Bloom,  1981), 
increasing  utilization  of  cost-effective  technology,  and  special  needs  of 
students  (Gage  &  Berliner,  1979).  Such  learning  approaches  are  generally 
defined  as  systems  in  which  the  learning  environment  is  organized  and  maintained 
from  feedback  derived  uniquely  from  each  individual  student  (Langer,  1978). 
Feedback,  in  turn,  is  generally  defined  as  anything  that  tells  the  student  if 
the  answer  is  right  or  wrong. 

Typically  feedback  is  an  adjunct  process  designed  to  adjudicate  differences 
between  the  intent  of  the  instructional  program  and  perceived  outcomes  of 
student  interaction.  Almost  invariably,  feedback  is  assumed  to  fol  1  ow  a 
response.  Moreover,  most  systems  assume  a  direct  and  predictable  effect  of  the 
feedback  provided. 

However,  the  assumption  of  direct  causality  between  an  instructional 
stimulus  (e.g.,  content,  feedback)  and  subsequent  cognitive  processing  is 
generally  unproven  (Hinne,  1982;  Bilodeau,  1969).  Validation  of  any  process 
based  upon  a  product  (l.e.,  a  response)  is  difficult,  since  one  has  to  prove 
there  was  no  other  way  the  student  could  have  derived  the  response  (e.g., 
alternative  strategies).  Indeed,  for  the  behaviorists  the  assumption  of  a 
conditioning  paradigm  with  deterministic  transformations  (e.g.,  generalization 
and  discrimination  gradients)  is  absolutely  critical.  However,  no  systematic 
body  of  information  exists  on  feedback  for  the  instructional  developer. 

Feedback  is  a  difficult  construct  to  validate,  primarily  because  a  number 
of  theoretical  positions  are  embedded  in  the  concept.  One  is  the 
incentive- informat ion  controversy.  Since  the  instructional  implications  of 
feedback  may  be  considered  originating  essentially  from  the  work  of  Thorndike 
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(Thorndike,  1932),  behaviorlsts  have  defined  It  as  the  consequences  of  a  given 
response  and  made  It  operationally  part  of  the  conditioning  paradigm  (Anderson  & 
Faust,  1973).  Thus,  one  could  speak  of  reward  and  punishment  as  confirming  and 
dlsconfl rating  consequences  respectively,  and  feedback  overall  In  terms  of  an 
Incentive  model  (Buss,  Braden,  Orgel,  &  Buss,  1956).  The  feedback  term 
Hknow1edge  of  results"  (KR )  can  also  easily  be  utilized  by  behavlorists.  If  the 
conflrmatl on/di sconflrmatl on  produces  concomitant  changes  In  probability  of 
response  through  conditioning.  Instructionally,  more  informative  KR  models  can 
be  maintained  within  the  conditioning  paradigm  (Anderson,  Kulhavy,  &  Andre, 
1971),  since  the  behavioral  position  does  not  deal  directly  with  the  input  of 
Information  on  incentive,  per  se.  Feedback  under  these  circumstances  becomes  a 
mechanistic  process  of  confirming-disconfi  raring,  modifying  thought  by 
strengthening  or  suppressing  associations.  The  cognitive  position,  on  the  other 
hand  (e.g.,  Atkinson  k  Wickens,  1971),  assumes  that  the  Information  provided 
contributes  directly  to  the  Immediacy  and  direction  of  change.  The  cognitive 
viewpoint  accepts  Idiosyncratic  transformation  of  data. 

Unfortunately  for  the  strictly  behavioral  position,  the  phenomenon  of 
subjective  organization  Including  the  use  of  strategies  In  recall  is  well 
established  (e.g.,  Anderson,  1980).  Indeed,  as  early  as  1932  Bartlett 
postulated  the  principle  of  constructive  memory.  Other  types  of  memory 
organizations  have  been  subsequently  derived  In  addition  to  subjective;  i.e., 
categorical  and  associative  organization  are  well  established  constructs. 

The  meaningful  ness  of  the  material  appears  to  Influence  recall,  although 
the  findings  have  not  established  a  clear  pattern  (e.g.,  Glidden  &  Roemer,  1974; 
Gorfeln  k  Blair,  1971).  However,  while  subjective  organization  may  vary  from 
Individual  to  Individual,  there  Is  stability  within  the  individual.  Thus 
Kozmlnsky,  Klntsch,  Coreu,  k  Bourne  (1979)  found  at  least  four  different 
strategies  In  a  moderately  complex  decision-making  problem,  while  Sternberg  and 


Ketron  (1982)  found  that  untrained  subjects  In  an  analogy  problem  solving  task 
tended  to  utilize  reliably  effective  self-generated  strategies.  These 
Individual  differences  lead  to  problems  Involving  linear  feedback  models,  since 
the  developer's  assumptions  about  what  Is  needed  may  differ  markedly  from  those 
of  the  learner  at  any  point. 

The  thesis  can  be  made  that  recent  advances  in  cognitive  science  have 
invalidated  many  earlier  behavioral  assumptions  about  learning.  To  the 
curriculum  developer,  particularly  in  learning  situations  depending  on  a  heavy 
feedback  component,  the  issues  have  grown  exceedingly  more  complex.  Behavioral 
management  systems  have  followed  Skinner's  dictum  of  small  controlled  steps  to 
reduce  error,  and  as  a  consequence  feedback  has  been  dictated  by  the  assumption 
that  the  strategy  minimizes  significant  individual  transformations  of  meaning. 

The  paradox  is  that  while  modern  cognitive  theory  can  be  described 
accurately  as  information-processing  (Bourne,  Dominowski,  &  Loftus,  1979),  the 
cognitive  theorists  have  not  resolved  problems  of  content  and  sequencing, 
especially  in  higher-order  learning  environments.  The  issue  of  feedback  is 
scarcely  addressed  In  Instructional  terms  (Clark,  1982).  Behavioral  approaches 
have  had  a  great  deal  to  say  about  content  and  sequencing  as  well  as  feedback, 
but  have  generally  Ignored  learner  transformations.  The  cognitive  position  sees 
the  informational  dimension  of  feedback  as  directive,  and  regards  the  incentive 
dimension  as  minimal  in  most  cases. 

The  cognitive  question  therefore  is  basically:  what  kinds  of  information? 
In  a  recent  study  prespecified  schemata  were  matched  to  the  organization  of  the 
content,  and  student  learning  was  enhanced  (Brooks  &  Dansereau,  1983). 

Typically  though,  classroom  Instructional  environments  do  not  operate  under  such 
precise  controls.  Thus,  the  feedback  may  assist,  hinder,  or  have  no  Impact  at 
all. 

However,  recognizing  the  necessity  of  Immediate  change  In  case  of  error. 
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the  behavioral  psychologists  have  developed  the  concept  of  "corrective" 
feedback,  defined  as  specifically  directing  the  learner  to  modify  an  Incorrect 
response  (Anderson  &  Faust,  1973).  Whereas  knowledge  of  results  (KR)  assumes 
that  probability  of  response  will  eventually  change  through  conditioning, 
corrective  feedback  mandates  changes. 

The  Incentive-Informational  dispute  is  probably  the  most  central  issue,  and 
has  not  been  resolved.  Indeed,  it  Is  likely  that  feedback  combines  both 
aspects,  although  the  experimenter  has  the  opportunity  to  suppress  or  enhance 
either  dimension  depending  on  design  bias  (Getsie,  1982). 

Another  major  problem  in  neatly  conceptualizing  feedback  is  that  it  is 
basically  a  multi-dimensional  concept  (Holding,  1965).  Using  a  branching 
paradigm.  Holding  postulated  several  dichotomies.  The  first  creates  a 
distinction  between  extrinsic  and  Intrinsic  feedback.  Each  of  the  first  pair, 
in  turn,  can  be  subdivided  Into  concurrent  and  terminal.  Subsequent  dichotomies 
Include:  Immediate-del ayed,  verbal -non-verbal ,  and  separate  (after  each 
response) -accumulated  (after  all  responses).  The  number  of  possible 
permutations  and  combinations  Is  awesome.  A  perusal  of  the  dichotomies  clearly 
Indicates  that  feedback  experiments  are  likely  to  persist  in  having  the 
directive  and  Incentive  results  confounded  (Locke,  Cartledge,  &  Kleppel,  1968), 
although  as  previously  suggested  by  appropriate  selection  of  treatment  variables 
the  incentive  or  Information  component  can  be  suppressed. 

Feedback,  regardless  of  conceptual  complexities  and  mixed  research  findings 
(e.g.,  Barringer  A  Gholson,  1979),  Is  considered  a  critical  component  by  most 
Instructional  systems  analysts  and  psychologists  (e.g.,  Davis,  Alexander,  & 
Yelon,  1974).  By  any  set  of  standards  It  is  considered  absolutely  essential  in 
designing  auto-tutorial  systems  Involving  computer-based  Instruction  (e.g., 
Gagne,  Wager,  A  Rojas,  1981).  The  reason  Is  that  under  conditions  of 
Individualized  Instruction  the  student  Is  severely  limited  In  terms  of  different 
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sources  of  feedback,  and  hence  the  “equalization  phenomenon"  (Hilgard  4  Bower, 
1975)  is  attenuated.  That  is,  the  student  cannot  make  up  for  deficiencies 
anywhere  in  the  instructional  situation  by  utilizing  a  variety  of  informational 
sources.  We  are  not  arguing  that  teachers  in  traditional  classrooms  are  better 
managers  of  feedback.  Rather,  in  individualized  technologically-assisted 
settings  the  student  is  obviously  more  restricted  in  terms  of  possible  sources 
of  assistance. 

Behavioral  psychologists,  as  well  as  those  interested  in  the  area  of 
artificial  intelligence  (AI)  have  not  been  unmindful  of  this  problem  (Bunderson 
A  Faust,  1976).  Given  the  early  failures  in  machine  language,  more  extensive 
attempts  were  made  to  develop  sophisticated  programs  which  were  capable  of 
carrying  out  a  tutorial  dialogue  with  the  student.  One  example  was  SOPHIE 
(Sophisticated  Instructional  Environment),  built  around  problem-solving 
electronic  circuitry  (Brown,  Burton,  &  Bell,  1974). 

The  basic  assumption  was  that  by  providing  detailed  and  analytical 
feedback,  errors  could  be  markedly  reduced,  and  students  redirected  in  their 
thinking  (Brown  et  al.,  1974).  While  students  may  attempt  to  correct  themselves 
with  repeated  opportunities  (e.g..  Singer  4  Pease,  1978),  most  Instructional 
psychologists  point  to  findings  of  error  perseveration  (e.g.,  Kaess  4  Zeaman, 
1960)  as  something  to  be  avoided  (e.g.,  Howe,  1970;  Ladas,  1980).  Ultimately, 
one  supposes  the  goal  Is  errorless  learning  (Terrace,  1963). 

Unfortunately,  recent  work  on  tutorial  languages  suggests  that  the 
elaborated  conceptual  model  may  be  detrimental  to  student  performance 
(Gallagher,  1981).  Using  the  BLOCKS  program  Gallagher  provided  diagnostic 
feedback  and  Information  based  on  an  "expert"  problem-solving  model  (which  is 
typical  of  tutorial  programs).  However,  students  not  given  this  help  actually 
performed  better  on  subsequent  problems.  Kulhavy  (1977)  has  suggested  that 
actually  the  more  complicated  feedback  programs  may  be  a  form  of  learning  task 
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In  addition  to  the  text.  Certainly  more  Is  not  always  necessarily  better. 

Sternberg  and  Ketron's  (1982)  previously  cited  work  also  suggests  that  a 
predetermined  feedback  strategy  model  may  actually  be  providing  the  learner  with 
Irrelevant  data  not  compatible  with  the  functional,  student-generated  strategy. 
It  is  Ironic  to  note,  when  early  assumptions  about  simplicity  and  directness  of 
feedback  in  programmed  Instruction  were  Invalidated  by  students  short-circuiting 
the  feedback  process  by  copying  answers  (e.g.,  Anderson  et  al.,  1971), 
subsequent  attempts  to  overcompensate  In  the  other  direction  may  also  have  had 
deleterious  effects.  The  key  lies,  of  course.  In  the  basic  assumptions  of  a 
feedback  model  based  on  an  "expert11  or  linear  strategy  versus  those  derived 
explicitly  from  an  analysis  of  the  Individual  learner's  strategy.  The 
difference  Is  expressed  also  in  terms  of  the  curriculum  model,  with  the 
curriculum  developer  on  one  hand  deciding  in  advance  what  should  be  done  as 
compared  to  a  more  functional  analysis  of  what  the  individual  learner  is  doing. 
While  the  research  evidence  supporting  either  experimenter  or  learner  determined 
sequences  Is  mixed  (Dansereau,  Evans,  Wright,  Long,  A  Atkinson,  1973),  there  is 
no  reason  for  solely  accepting  the  developer  determined  model. 

There  are  a  number  of  other  issues  worth  noting.  Foremost  is  the  finding 
that  dlsconfi nnatlon  (negative)  feedback  Impacts  achievement  much  more  than 
confirmation  (positive)  feedback  (Kulhavy,  1977).  Researchers  In  concept 
acquisition  (e.g.,  Bruner,  Goodnow,  A  Austin,  1956;  Trabasso  A  Bower,  1968) 
have  found  that  learners  shift  strategy  after  discontinuation,  although  not  back 
to  some  simple  random  selection  (Levine,  1966).  Generally,  the  more  subtle 
and/or  qualitative  aspects  of  feedback  have  been  Ignored  (Kulhavy,  1977). 

The  use  of  feedback  during  and  after  the  acquisition  phase  presents  several 
problems.  If  It  Is  given  too  soon  or  Is  too  readily  accessible,  the  error  rate 
Increases  (Sullivan,  Schutz,  A  Baker,  1967).  Indeed,  several  researchers  have 
allowed  the  subject  to  determine  when  feedback  Is  to  be  given  (Anderson  et  al.. 
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1971;  Mel  chi ng,  1966).  The  findings  show  no  clear-cut  superiority,  but  suffer 
from  some  methodological  analysis  problems  which  will  be  discussed  later. 

Moreover,  delay  In  knowledge  of  results  also  has  diverse  effects.  Kulhavy 
(1977)  has  argued  that  while  feedback  should  be  provided  as  often  as  possible 
during  Instruction,  the  content  organization  and  difficulty  of  the  material  is 
also  crucial.  If  the  material  is  very  difficult,  students  spend  most  of  their 
time  guessing  at  answers  and  trying  to  match  feedback  to  the  question  (Kulhavy, 
Yekovich,  &  Dyer,  1976).  However,  where  the  material  is  relatively  easy  and/or 
student  confidence  is  high,  students  spend  relatively  little  time  on  feedback. 
Indeed,  Kulhavy  (1984)  currently  is  concentrating  on  the  confidence  issue. 
Battig  (1966),  however,  suggested  that  difficulty  in  learning  might  have 
positive  consequences.  Certainly  the  issues  of  when  and  how  much  feedback  are 
significant  in  problems  of  transfer. 

Moreover,  delay  in  feedback  seems  to  sometimes  facilitate  acquisition 
(e.g.,  Surber  &  Anderson,  1975).  The  argument  is  that  the  delay  allows  for 
Incorrect  responses  to  be  forgotten,  which  approximates  the  practice 
Interference  paradigm  (Kulhavy  &  Anderson,  1972).  Finally,  very  p  rtinent  to 
our  theoretical  position  and  use  of  feedback  is  a  study  by  Kulhavy  and  Parsons 
(1972)  which  suggests  that  the  effect  of  feedback  is  minimal  unless  the 
instructional  content  is  organized  in  some  way  meaningful  to  the  learner  (e.g., 
schemata,  script,  etc.). 

The  notion  of  text  organization  appears  to  be  critical.  Researchers 
Investigating  the  processing  of  prose  have  assumed  that  the  key  problem  for  the 
reader  Is  to  enhance  meaningfulness,  i.e.,  coding.  The  text  analysis  approach 
has  developed  within  a  number  of  specialized  subareas,  including  grammar, 
content,  orienting  tasks,  and  knowledge  bundles.  At  the  present  moment,  the 
theoretical  trend  is  epistemological  (i.e.,  how  knowledge  Is  structured). 

Current  theories  have  been  tied  essentially  to  measures  of  recall  (Reder,  1980). 


Research  has  clearly  demonstrated  that  recall  for  prose  Is  subject  to  the 
logic  of  the  discourse  (e.g..  Prase,  1969;  Kulhavy,  Schmid  &  Walker,  1977),  as 
well  as  the  serial  position  of  the  discourse  elements.  There  have  been  a  number 
of  attempts  to  characterize  text  processing  (e.g.,  Dawes,  1964;  Crothers,  1972; 
Frederlcksen,  1975;  Rumelhart,  1977;  Meyer,  1975).  Most  have  been 
characterized  as  more  or  less  successful  within  the  limits  Imposed  by  the 
specific  model  (Reder,  1980).  Generally  these  programs  attempted  to  simulate 
via  computer  the  sequential  reorganization  of  prose  during  processing.  Sequence 
of  proposition  occurrence  Is  an  explicit  parameter  In  one  of  the  most  important 
theories  of  text  processing  (Kintsch  &  van  Dljk,  1978). 

More  recently,  attempts  have  been  made  to  link  acquisition  to  outcomes  of 
schemata  (or  related  constructs).  These  are  usually  defined  as  abstract 
representations  of  knowledge  within  the  memory  structure  of  the  learner  (e.g., 
Rumelhart  &  Ortony,  1977),  and  are  considered  an  integral  part  of  text 
processing.  The  Brooks  and  Dansereau  (1984)  study  Is  a  typical  example  of  this 
approach.  It  Is  assumed  that  Input  can  be  facilitative  or  Interfering  depending 
on  the  interaction  between  schemata  and  discourse  logic. 

Subjective  organization,  as  analyzed  In  text  processing.  Is  also  a  function 
of  familiarity  and  meaningful  ness.  Indeed  some  researchers  (e.g.,  Anderson  & 
Reder,  1979)  argue  that  the  schemata  available  to  the  subject  may  have  more 
Impact  on  acquisition  than  the  logic  of  the  discourse  Itself. 

While  considerable  progress  has  been  made  using  well  formed  texts, 
comparatively  little  systematic  research  has  been  done  using  disconnected 
discourse.  This  Is  surprising  since  the  lack  of  apparent  content  organization 
would  tend  to  equalize  subjects  Initially  regardless  of  prior  experiences  with 
the  materials.  One  would  assume  under  these  circumstances  the  Impact  of 
feedback  should  be  maximized,  since  subjects  should  need  help  to  develop  the 
logic  of  the  passage.  In  most  feedback  studies,  the  procedure  has  been  to 
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develop  the  discourse  logic  so  tightly,  that  Individual  differences  In 
processing  are  minimized.  This  may  be  confounding  the  feedback  problem. 
Generally,  the  findings  with  disconnected  discourse  'ack  coherence  In  terms  of 
outcomes.  Overall,  using  a  free  recall  procedure,  scrambled  sequences  generally 
yield  poorer  performance  than  connected  discourse,  regardless  of  unit  size 
(Dansereau  et  al.,  1973). 

In  many  experiments  involving  scrambled  presentation  little  is  done  to 
assist  the  learner  using  disconnected  materials.  In  a  typical  study  (Frase, 
1969),  subjects  using  a  name  concept  strategy  were  superior  to  those  using  an 
attribute  strategy,  and  both  were  superior  to  the  scrambled  order  group.  Sasson 
(1971)  interrelated  two  different  topics,  a  variant  of  the  scrambled  order 
method,  and  found  in  both  instances  that  a  thematic  organization  produced 
superior  student  achievement  as  compared  to  temporal  (i.e.,  sentences  included  a 
date)  and  logical  arrangements. 

Meaningful  ness  of  the  materials  has  also  been  investigated.  Bruning  (1970) 
embedded  the  test  items  in  contexts  which  were  relevant  and  ordered,  relevant 
and  scrambled,  and  irrelevant  and  scrambled.  The  findings  are  somewhat  unusual 
in  that  there  was  no  significant  difference  between  the  relevant  and  ordered  and 
the  relevant  and  scrambled  contexts,  but  both  were  superior  to  the  irrelevant 
and  scrambled.  As  in  nearly  all  these  studies  the  discussion  dealt  with  concept 
acquisitions  in  the  ordered  material;  no  one  seemed  to  pay  much  attention  to 
the  what  and  how  of  achievement  in  the  scrambled  treatment.  Tobias  (1973)  using 
programmed  instruction,  found  that  familiarity  overcame  to  some  extent  the 
effects  of  scrambling.  Again  using  programmed  instruction.  Brown  (1970)  found 
that  students  encountering  a  scrambled  mathematics  sequence  took  more  time,  made 
more  errors,  and  performed  more  poorly  on  problems.  This  is  to  be  expected  in  a 
subject  area  in  which  meaning  is  clearly  tied  to  a  hierarchical  sequence. 

The  interrelatedness  of  the  sentences  is  also  a  factor.  Kissler  and  Lloyd 


(1973)  found  that  highly  Interrelated  sentences  were  more  sensitive  to 
scrambling,  and  argument  overlap  Is  an  Important  part  of  the  Kintsch  and  vanDijk 
(1978)  definition  of  macroproposition.  Rothkopf  (1962)  Indirectly  assessed  this 
dimension  when  he  used  loosely  organized  sentences  to  create  his  discourse 
treatments.  Anderson  (1966)  suggested  that  programmed  Instructional  units 
(I.e.,  the  frames)  are  developed  to  be  independent,  and  with  the  high  redundancy 
of  programmed  Instruction  units  scrambling  may  not  be  as  destructive  of 
performance.  In  a  more  recent  study,  Taylor  and  Samuels  (1983)  found  that 
children  who  were  aware  of  text  organization  recalled  more  of  normal  passages 
than  scrambled,  but  children  who  were  unaware  of  organization  showed  no 
differences  between  normal  and  scrambled  passages.  Unfortunately,  as  in  many  of 
these  studies,  the  text  was  artificially  structured.  One  wonders  if  the 
contrived  logic  did  not  interfere  with  the  strategies  usually  employed  by  some 
children. 

While  none  of  the  studies  cited  a  feedback  mechanism,  a  study  done  by 
Schultz  and  OiVesta  (1972)  Is  unique  in  that  students  were  allowed  to  take  notes 
(an  outside  confirmation  source).  Again,  while  the  emphasis  was  on  the 
strategies  employed  with  respect  to  organized  materials,  the  researchers  also 
observed  that  students  taking  notes  modified  the  passages  to  suit  their  needs. 
That  is,  they  were  not  forced  to  at  least  Initially  accept  the 
experimenter-determined  logic. 

In  summary,  Bartlett  (1932)  demonstrated  that  strong  recall  tended  to  fit 
schemata.  Current  research  (e.g.,  Kintsch  &  vanDijk  1978;  Meyer,  1975)  has 
reaffirmed  this  proposition  by  demonstrating  that  the  probability  of  inclusion 
within  recall  is  related  to  the  importance  of  the  statement  to  the  theme. 
Instructional ly,  the  Implications  are  significant.  In  that  organization  appears 
to  be  modified  at  time  of  Input  (Mandler,  1972).  Hence,  relevancy  is  a  function 
of  learner  schemata.  Behaviorally  oriented  instructional  psychologists  can 
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develop  algorithms  for  organizing  concept  elements  (e.g.,  Merrill  &  Tennyson, 
1979)  only  by  Ignoring  the  mechanism  of  chunking  (Miller,  1956;  Buschke,  1976). 
Since  It  is  assumed  that  feedback  Involves  active  memory  (Anderson,  1980),  both 
timing  as  well  as  congruence  of  feedback  to  schemata  and  strategy  are  crucial. 

It  can  be  argued  that  in  most  educational  situations,  particularly  In 
higher-order  and  non-hierarchical  learning,  feedback  might  better  serve  by 
assisting  In  self-correction  (Anderson,  1980).  This  avoids  the  problem  of  the 
Instructional  developer  intruding  into  functional  and  perhaps  unique  memory  and 
processing  strategies.  Certainly  we  should  take  heed  from  Nisbett  and  Wilson's 
(1977)  findings  that  learners  often  cannot  accurately  verbalize  their  own 
strategies.  If  they  cannot  do  it,  then  instructional  psychologists  may  Indeed 
be  on  dangerous  conceptual  grounds.  Hence,  while  Singer  (1980)  called  for 
optimizing  feedback,  in  the  end  it  may  be  the  learner  that  dictates  the  meaning 
of  “optimal."  This  research  begins  to  address  such  issues. 

Procedures 

The  subjects  were  64  Fall  *83  undergraduate  students  from  the  Introductory 
Psychology  course  at  the  University  of  Colorado.  All  students  In  the  course  are 
required  to  spend  several  hours  as  subjects  in  psychological  experiments  as  part 
of  the  course  requirements.  The  subjects  were  given  the  task  on  an  individual 
basis,  with  order  of  treatments  randomly  assigned. 

The  passages  selected  were  broken  down  and  ordered  by  sentence.  Within  the 
passage  each  sentence  was  typed  separately  on  a  3x5  index  card.  To  assist 
subjects  in  ordering  the  sentences,  a  wooden  board  with  35  slots  was 
constructed.  Each  slot  provided  space  for  one  card.  Cards  had  to  be  picked  up 
one  at  a  time,  and  placed  In  the  slots.  This  forced  serial  processing. 

The  subjects  were  Informed  that  the  purpose  of  the  experiment  was  to 
determine  how  people  made  meaning  from  material  they  read,  particularly  how 
meaning  Is  constructed.  To  familiarize  the  subjects  with  the  experimental  task. 
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they  first  performed  a  practice  task  which  consisted  of  ordering  11  sentences 
comprising  the  story  of  "The  Goose  that  Laid  the  Golden  Egg."  As  In  the 
experiment,  each  card  contained  a  single  sentence,  with  the  order  of  cards 
scrambled.  The  subjects  were  told  to  reorder  the  sentences  to  make  the  best 
complete  story,  and  signal  when  they  felt  the  story  was  complete.  All  questions 
regarding  procedures  were  answered  at  this  time. 

If  the  subject  was  to  receive  the  feedback  treatment  on  the  experimental 
passage  to  follow,  the  experimenter  gave  the  subject  five  tokens  to  be  used  for 
feedback  during  the  practice  task.  The  experimenter  Informed  the  subject  that 
by  giving  up  a  token,  the  subject  would  be  told  If  a  card  had  been  placed 
correctly.  Thus,  they  could  ask  for  help  a  maximum  of  five  times  if  they  needed 
It.  The  subject  was  told  that  a  "yes"  meant  that  the  card  In  question  was 
correctly  placed  with  respect  to  the  card  Immediately  preceedlng  It.  Any  other 
placement  was  considered  a  "no".  For  example,  suppose  the  subject  picked  up 
card  #7  In  the  preassigned  order.  If  the  card  was  placed  directly  after  card 
#6,  the  move  was  designated  as  correct;  any  other  placement  was  wrong.  The 
no-feedback  subjects  were  not  given  any  assistance. 

Following  the  practice  task,  the  subjects  were  given  one  of  four  scrambled 
packets  of  cards.  The  passages  were  drawn  from  two  sections  of  Laura  Fermi's 
account  of  the  development  of  the  first  atomic  bomb  (Gardner,  1957).  Throughout 
our  discussion,  the  passages  will  be  referred  to  as  A  and  6.  Both  passages  were 
related  to  the  development  of  the  atomic  bomb,  although  the  phrase  "atom  bomb" 
never  appears  per  se  In  the  passages. 

For  each  passage  a  13  and  a  19  sentence  version  was  prepared.  The 
sentences  except  for  very  minor  modifications  were  verbatim,  with  the  order 
following  that  of  the  text  author.  The  19  sentence  passage  Included  the  13 
sentence  version  plus  an  additional  six  sentences  from  passages  Immediately 
following  or  preceding  the  smaller  version.  Hence,  the  four  passages  are 


designated  A-13,  A-19,  B-13,  and  B-19.  For  each  passage,  two  randomized  orders 
were  used,  although  each  subject  used  only  one  order.  Subjects,  therefore, 
received  a  scrambled  packet  of  13  or  19  cards,  with  one  of  two  contents. 

As  Indicated  earlier,  the  subjects  were  assigned  to  either  a  feedback  or 
no-feedback  condition.  The  feedback  subjects  received  five  tokens  once  more, 
while  the  no-feedback  subjects  again  reconstructed  a  passage  without  assistance. 
The  subjects  signalled  when  complete.  Immediately  after  the  cards  had  been 
ordered,  the  subjects  were  tested  on  recall  and  recognition  in  that  order.  For 
the  free  recall  task,  the  subjects  were  asked  to  "write  down  as  many  sentences 
as  possible;  they  need  not  be  verbatim  or  in  any  special  order."  For  the 
recognition  task,  the  subjects  were  given  a  test  consisting  of  13  or  19  pairs  of 
sentences  (depending  on  which  length  they  had  been  assigned);  within  each  pair, 
one  sentence  was  an  original  while  the  other  was  a  paraphrase.  The  subjects 
were  asked  to  choose  the  original  sentence. 

The  Independent  variables  were  content  (CO),  number  of  sentences  (NS),  and 
feedback  condition  (FB).  The  dependent  variables  were  time  to  order  the  passage 
(TM),  number  of  moves  (MV),  number  of  tokens  used  (TK),  concordance  (TA), 
percent  recall  scores  (RL),  and  percent  recognition  scores  (RG). 

Feedback  as  a  conflrmation-disconflrmatlon  rationale  has  been  explained.  A 
“move"  was  construed  as  changing  the  relative  position  of  a  card  to  one  or  more 
other  cards.  For  example,  consider  three  cards  In  sequence:  #1,  #2,  and  #3. 

If  Card  #1  was  placed  between  #2  and  #3,  or  after  #3,  either  change  was  counted 
as  one  move. 

The  starting  and  stopping  times  were  recorded  to  give  a  measure  of  time, 
and  the  number  of  tokens  used  was  also  recorded.  In  addition,  while  the 
subjects  took  the  recall  and  recognition  tests,  the  experimenter  recorded  the 
sentence  order  in  which  the  subject  had  placed  the  cards.  Kendall's  Tau  was 
used  to  obtain  a  measure  of  concordance  with  the  original  text  order.  The 
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recall  and  recognition  scores  were  derived  from  the  written  materials  and  paired 
sentences  respectively. 

Analysis 

The  experimental  design  was  a  factorial  2  (feedback  -  no  feedback)  x  2 
(number  of  sentences:  13  -  19)  x  2  (content:  A  -  B).  All  cells  were  equally 
balanced  as  to  number  of  subjects.  To  repeat,  the  dependent  measures  were  X 
recall  (RL),  %  recognition  (RG),  time  (TM)  In  minutes,  moves  (MV),  concordance 
(TA),  and  number  of  tokens  (TK).  The  Independent  variables  were  feedback  (FB), 
number  of  sentences  (NS),  and  content  (CO). 

Prior  to  the  scoring  of  the  recall  tests,  the  selections  were  broken  down 
Into  idea  units,  following  the  work  of  Bovalr  and  Kieras  (1981).  Table  1  gives 
the  number  of  Idea  units  for  each  passage. 


Insert  Table  1  about  here 


Although  the  passages  were  taken  from  the  same  short  chapter  by  Fermi,  if 
the  number  of  Idea  units  can  be  construed  as  a  measure  of  cognitive  complexity, 
the  "A"  material  appeared  to  differ  from  "B".  The  significance  of  the 
difference  was  obviously  an  empirical  Issue.  Both  passages,  however,  followed 
the  same  narrative  style  of  the  author.  In  order  to  compare  recall  for  the 
different  numbers  of  ideas  In  the  four  passages,  scores  were  converted  to 
percentages,  and  are  reported  as  such  throughout. 

The  first  analysis  employed  was  MANOVA,  using  TM,  MV,  TA,  XRL,  and  XRG  as 
dependent  measures.  Table  2  presents  the  MANOVA  data.  All  the  appropriate  F 
values  reported  are  Hotelllngs'  coefficients. 


Insert  Table  2  about  here 
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Interestingly,  there  were  no  Interaction  effects.  Table  3  presents  the 
univariate  Fs  for  the  dependent  variables,  main  effects  only. 


Insert  Table  3  about  here 


The  means  and  standard  deviations  for  the  dependent  variables  are  given  in 
Table  4. 


Insert  Table  4  about  here 


From  the  pattern  of  significant  univariate  F  tests  as  a  guide,  several 
trends  became  clear.  With  feedback,  the  means  for  TM  (13.2  minutes)  and  MV 
(12.3  minutes)  differ  significantly  from  the  no-feedback  condition  (9.0  and  8.8 
minutes,  respectively).  These  results  are  to  be  expected,  since  subjects  having 
feedback  available  would  be  expected  to  make  more  moves  and  use  more  time.  The 
simple  r  between  time  and  moves  Is  .74,  which  Is  significant  at  the  .01  level. 

Within  number  of  sentences,  the  19  sentence  passage  means  for  TM  and  MV  are 
higher  than  13  sentence  passage  means  (7.3  and  6.6  versus  15.2  and  14.4, 
respectively).  Longer  passages  should  require  more  processing  time,  and  this  is 
reflected  in  the  data.  As  one  might  also  expect,  the  mean  for  TA  (concordance) 
Is  higher  for  the  shorter  passages  as  compared  to  the  longer  ones  (.63  and  .29 
respectively).  As  the  size  of  the  passage  Increases,  the  likelihood  that  the 
subject  will  derive  an  order  exactly  like  the  original  diminishes,  regardless  of 
feedback.  It  began  to  appear  that  passage  length  should  be  treated  as  a  random 
variable,  which  has  not  been  the  customary  procedure  In  the  past.  Finally,  *  RL 
was  higher  for  the  shorter  passages  (48.2  vs  39.3);  although  non-significant,  % 
RG  followed  the  same  trend. 

The  main  effect  form  for  number  of  sentences  (NS)  had  Its  counterpart  In 
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content  (CO).  Unlike  our  findings  for  NS,  TM  and  MV  were  not  significant. 
However,  there  were  significant  differences  for  TA,  %  RL,  and  %  RG. 

As  observed  In  Table  1,  If  Idea  units  are  a  measure  of  complexity,  the  "B" 
passage  should  have  been  considered  conceptually  simpler.  In  the  case  of  TA, 
the  mean  for  “BM  (.52)  is  higher  than  for  ”A"  (.40).  Given  a  simpler  passage, 
achieving  concordance  should  be  easier.  What  Is  not  obvious  Is  that  the 
percentage  scores  for  RL  and  RG  are  higher  for  the  more  difficult  "A"  passage 
(47.1  versus  40.1  and  85.1  versus  76.6)  respectively. 

To  test  our  assumptions  regarding  the  criticality  of  number  of  sentences 
and  content,  we  reanalyzed  the  data  with  TM,  MV,  and  TA  as  covariates.  The 
MANCOVA  findings  are  given  in  Table  5.  All  approximate  F  values  reported  are 
Hotel  lings. 


Insert  Table  5  about  here 


Again,  there  were  no  interaction  effects.  There  were  main  effects, 
however,  for  NS  and  CO.  Paralelling  our  MANOVA  analyses,  the  univariate  F 
values  for  X  RL  and  X  RG  within  NS  and  CO  are  given  in  Table  6. 


Insert  Table  6  about  here 


The  means  and  standard  deviations  for  X  RL  and  X  RG  within  CO  and  NS  are 
given  in  Table  7. 


Insert  Table  7  about  here 


MANCOVA  yielded  a  pattern  of  results  similar  to  that  for  MANOVA.  Within 
number  of  sentences  the  13-sentence  passage  X  RL  and  X  RG  mean  scores  are  higher 
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(47.4  and  83.0  versus  39.3  and  79.2  respectively).  Within  content,  the  "A" 
passage  %  RL  and  X  RG  mean  scores  are  higher  than  HBU  (47.1  and  85.1  versus  39.6 
and  77.1  respectively).  Findings  for  higher  recognition  than  recall  scores  are 
consistent  with  the  literature. 

However,  at  the  moment  we  can  only  speculate  why  the  more  complex  “A" 
passage  produced  higher  recall  and  recognition  scores.  There  are  no  significant 
interaction  effects,  and  both  sets  of  analyses  lead  to  the  Inescapable 
conclusion  that  passage  length  and  content  must  be  treated  as  random  variables. 

Finally,  we  decided  to  examine  the  impact  of  number  of  tokens  used  (TK). 
Previous  studies  involving  student  choice  of  feedback  did  not  take  into  account 
frequency  of  subject  usage.  Correlating  TK  against  TM,  MV,  X  RL,  XRG,  and  TA 
yielded  no  significant  findings.  Furthermore,  the  mean  number  of  tokens  used 
did  not  reflect  passage  length.  These  data  are  given  in  Table  8. 


Insert  Table  8  about  here 


Subjects  dealing  with  the  19  sentence  passages  tended  to  use  fewer  tokens, 
although  the  differences  are  not  statistically  significant.  The  absence  of 
significant  differences  is  not  surprising  in  light  of  the  MANOVA  and  MANCOVA 
findings. 

If  feedback  did  not  yield  statistical  differences,  did  it  have  any  Impact 
at  all?  A  secondary  analysis  involving  the  protocols  (l.e.,  subject-assigned 
order  of  sentences)  shows  some  interesting  trends. 

Arbitrarily  we  divided  the  13-sentence  passages  into  two  units,  consisting 
of  sentences  1-6  and  7-13.  For  the  19-sentence  passages,  the  divisions  were 
sentences  1-6,  7-13,  and  14-19.  For  passage  A-13,  none  of  the  subjects  in  the 
non-feedback  condition  put  the  first  six  sentences  in  the  preassigned  order. 

The  maximum  occurrence  of  sentences  from  the  first  six  in  the  first  six  slots. 


without  respect  for  order.  Is  48  (l.e.,  8  subjects  x  6  sentences).  Protocol 
analyses  Indicate  that  37  of  the  48  slots  Indeed  consisted  of  Sentences  1-6 
(77. IX)  In  any  order.  This  means  there  were  11  Intrusions  (l.e..  Sentences 
7-13).  For  Sentences  7-13,  none  were  In  the  preassigned  order.  Of  the  56  slots 
available,  45  (80. 4%)  consisted  of  Sentences  7-13.  Again,  there  were  11 
Intrusions,  consisting  of  Sentences  1-6  not  previously  placed  correctly. 

The  A-13  feedback  group  displayed  somewhat  different  results.  Two  subjects 
exactly  reproduced  the  preassigned  order  for  Sentences  1-6.  Again,  with  a 
potential  of  48  card  placements,  33  (68. 8%)  included  Sentences  1-6.  This  is 
quite  startling,  given  the  fact  that  two  subjects  exactly  reproduced  the 
preassigned  order;  the  other  subjects  seemed  to  have  been  hindered  by  feedback. 
That  Is,  two  subjects  accounted  for  12  cards,  while  the  other  6  accounted  for 
21.  Moving  on  to  Sentences  7-13,  we  find  41  (73.2%)  correctly  grouped.  Again, 
the  15  Intrusions  represent  the  incorrectly  placed  1-6  sentences.  However, 
regardless  of  degree  of  agreement,  there  were  no  differences  in  X  RL  and  %  RG 
scores  attributable  to  concordance  with  the  preassigned  order.  Table  9  gives 
the  data  for  all  the  passages. 


Insert  Table  9  about  here 


While  the  number  of  sentences  used  per  unit  In  the  protocol  analyses  was 
arbitrary,  two  trends  appeared.  The  first,  regardless  of  feedback  length, 
concordance  generally  diminishes  with  passage  length.  Subjective  organization 
seems  to  take  over.  Second,  there  are  significant  content  differences.  For 
example,  under  the  no-feedback  condition,  subjects  tested  with  the  6-13  passage 
correctly  grouped  sentences  within  the  two  units,  although  no  single  subject 
reproduced  the  preassigned  order.  One  could  also  speculate  that  In  the  feedback 
conditions,  the  two  subjects  who  correctly  reproduced  sentence  order  for  1-13 


used  feedback  most  efficiently.  Certainly,  the  protocol  analyses  further 
emphasized  the  significance  of  passage  content  and  length,  as  well  as  the 
Idiosyncratic  nature  of  processing. 

The  unique  nature  of  processing  can  also  be  demonstrated  by  correlating  TA 
with  %  RC  and  %  RG  scores.  The  simple  rs  of  .25  and  -.11  are  non-significant. 

A  basic  assumption  of  most  text  processing  generally  follows  some  set  of 
universal  principles  such  as  serial  order.  In  this  study,  agreement  with  serial 
order  of  the  original  did  not  seem  necessary  for  understanding. 

Discussion 

Over  a  decade  ago  Clark's  classic  article  (1973)  warned  about  the  treatment 
of  content  as  a  fixed-effect.  Dealing  with  the  Issue  of  semantic  memory,  he 
suggested  that  content  also  be  considered  a  random  variable,  along  with 
subjects.  Unfortunately,  his  warnings  have  been  generally  ignored,  at  least  in 
the  literature  on  Instructional  use  of  feedback. 

Nevertheless,  the  findings  of  this  text  synthesis  approach  suggest  that  a 
major  problem  In  the  feedback  area  has  been  the  Issue  of  content.  Almost 
Invariably  the  experimental  procedure  has  been  to  construct  a  highly  organized 
passage,  or  series  of  passages,  following  a  predetermined  logical  algorithm. 
Generally,  material  developed  by  a  curriculum  writer,  whose  experiential 
background  Is  probably  significantly  different  from  Inexperienced  readers.  Is 
considered  acceptable  If  It  "makes  sense"  to  the  author. 

Given  this  sequenced  set  of  materials,  the  reader  has  no  choice  but  to 
determine  at  the  outset  the  logical  structure,  and  to  use  feedback  to  match 
Initial  and  developing  estimates  of  the  author's  Intent.  Essentially,  the 
reader  must  modify  whatever  Idiosyncratic  strategies  they  have  available.  The 
conflict  Is  between  subjective  organization  and  predetermined  logic,  and  the 
criticality  to  the  reader  of  variances  with  respect  to  the  original  logic. 

Since  conflrmatlon/dlsconflrmatlon  Is  defined  with  respect  to  a 


predetermined  model,  one  can  wonder  about  the  depth  of  processing.  Certainly 
findings  of  short-circuiting  of  the  instructional  process  (e.g.,  Anderson  et 
al.,  1971)  may  reflect  something  about  depth  and  commitment  to  processing.  It 
Is  no  wonder  that  feedback  may  hinder  as  well  as  have  no  discernible  impact  on 
retention.  Feedback  will  assist  to  the  extent  either  the  predetermined  logical 
model  has  been  correctly  determined,  or  congruence  with  the  learner  strategies. 

The  text  synthesis  method,  we  feel,  more  clearly  reveals  this  problem.  In 
the  text  synthesis  method,  the  reader  is  dependent  early  upon  internal  knowledge 
resources,  since  the  initial  logic  is  not  immediately  apparent.  As  the 
reconstruction  goes  on,  the  reader  can  establish  through  feedback  whether  the 
logic  used  is  compatible  to  that  of  the  author.  That  is,  is  the  reader's 
developing  comprehension  ever  brought  into  question  by  what  follows?  In  this 
study  concordance  was  not  a  factor  in  recall  or  recognition  scores,  which 
suggests  that  subjects  had  organized  the  passages  in  idiosyncratic  but 
acceptable  ways,  i.e.,  organizations  which  did  not  impact  subsequent 
performance.  Analyses  of  the  protocols  yielded  clusterings  and  "runs"  of 
sentences,  and  these  appeared  adequate  for  the  task.  These  findings  directly 
challenge  feedback  approaches  which  assume  an  "overkill"  is  necessary  or 
desirable. 

Feedback,  as  we  have  argued,  is  usually  designed  to  meet  instructional 
intent  with  too  little  consideration  of  reader  need.  Obviously,  giving  the 
reader  control  over  when  feedback  is  to  be  utilized  provides  some  flexibility, 
in  that  the  user  is  not  bombarded  with  unusable  information  at  inappropriate 
times.  Also,  a  functional  subjective  organization  can  be  allowed  to  develop, 
i.e.,  functional  in  terms  of  the  criterion  task. 

However,  the  type  of  feedback  may  also  have  to  vary  (e.g.,  Kulhavy,  1984). 
That  is,  this  study  used  a  simple  conflrmation/disconfl rotation  approach;  it  may 
be  that  the  reader  should  have  available  options  regarding  the  kinds  of  feedback 
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needed— a  type  of  menu.  The  text  synthesis  approach  enables  the  experimenter  to 
explore  both  timing  and  menu  much  more  sensitively  than  current  fixed-logic 
approaches. 

Instead  of  expending  time  on  a  priori  passage  logic  and  complex  linear 
model  feedback  aids,  a  better  approach  might  be  to  derive  feedback  alternatives 
from  the  text  synthesis  approach  and  let  the  reader  make  the  appropriate  choice. 
In  addition,  the  subject-controlled  menu/timing  approach  could  be  explored  in 
terms  of  such  variables  as  delayed  retention,  transfer,  type  of  criteria  task, 
cognitive  complexity  (i.e.,  content),  etc.  From  these  findings,  principles  may 
be  derived  to  assist  instructional  development.  The  findings  of  this  study 
appear  congruent  with  current  work  in  cognitive  psychology,  and  considerably  at 
variance  with  most  associative  feedback  models. 


Footnote 
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¥ 

FB  x  NS 

.14 

1.41 

ns 

j 

FB  x  CO 

.16 

1.63 

ns 

f* 

NS  x  CO 

.16 

1.66 

ns 

i 

FB  x  NS  xCO 

.13 

1.32 

ns 

i 

i 

*df  -  5,52 


MANOVA;  Univariate  Fs  for  Dependent  Variabl 
Dependent 


Condition 


Variable  F 
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Table  4 

Group  Means  and  (Standard  Deviations) 


Condition 

TM  (in  min.) 

MV 

TA 

XRL 

XRG 

Feedback 

13.2  (6.7) 

12.3  (8.1) 

.50  (.29) 

44.8  (16.9) 

81.2  (9.9) 

No  feedback 

9.0  (5.9) 

8.8  (6.3) 

.41  (.32) 

42.7  (16.4) 

80.3  (10.0) 

13  sentences 

7.3  (3.3) 

6.6  (4.1) 

.63  (.21) 

48.2  (16.2) 

83.2  (10.5) 

19  sentences 

15.2  (6.7) 

14.4  (7.9) 

.20  (.28) 

39.3  (16.0) 

79.2  (9.2) 

Selection  A 

11.5  (7.3) 

10.7  (7.0) 

.40  (.28) 

47.1  (17.7) 

85.1  (7.2) 

Selection  B 

11.0  (5.8) 

10.3  (7.9) 

.52  (.31) 

40.1  (15.3) 

76.6  (10.6) 
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Table  5 

MANCOVA;  Time,  Moves,  and  Tau  as  Covariates 

Approximate 


Variable 

Value 

F 

P 

FB* 

.005 

.13 

ns 

NS 

.34 

8.71 

.001 

CO 

.29 

7.54 

.001 

FB  x  NS 

.07 

2.25 

ns 

FB  x  CO 

.02 

.58 

ns 

NS  x  CO 

.02 

.51 

ns 

FB  x  NS  x  CO 

.07 

1.69 

ns 

*df  *  2,52 
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Table  6 

MANCOVA:  Univariate  Fs  for  %  Recall  and  %  Recognition 


Condition 

Variable 

F 

P 

CO 

%RL* 

4.38 

.04 

1RG 

11.18 

.002 

NS 

%RL 

5.28 

.03 

%RG 

12.71 

.001 

*df  =  1,53 


0 
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Table  7 

Recall  and  Recognition  Means  and  (Standard  Deviation) 

For  Passage 

Variable  Condition 


CO 

A 

B 

XRL 

47.1  (16.1) 

39.6  (13.6) 

XRG 

85.1  (7.2) 

77.1  (9.2) 

« 

13  NS 

19 

XRL 

47.4  (15.3) 

39.3  (16.0) 

XRG 

83.0  (8.8) 

79.2  (9.2) 

liiriiifltiihlrtni  iir  ■  rn 
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Table  9 

Agreement  with  Preassigned  Order 

Sentences  Sentences  Sentences  #  of  Ss  giving 

Passage  1-6  7-13  14-19  preassigned  order 


A- 13  FB  33/48  (68.8%)  41/56  (73.2%)  2 

A-13  No  FB  37/48  (77.1%)  45/56  (80.4%)  0 

B-13  FB  48/48  (100%)  56/56  (100%)  0 

B-13  No  FB  48/48  (100%)  56/56  (100%)  0 

A-19  FB  17/48  (35.4%)  21/56  (35.6%)  23/48  (47.9%)  0 

A-19  No  FB  16/48  (33.3%)  28/56  (50.0%)  17/48  (35.4%)  0 

B-19  FB  37/48  (77.1%)  31/56  (55.4%)  24/48  (50.0%)  0 

B-19  No  FB  27/48  (56.3%)  17/56  (30.4%)  11/48  (22.9%)  0 
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Appendix  A 
Version  A-13 

1.  A  shipment  of  crates  arrived  at  the  Chicago  stadium  on  a  Saturday. 

2.  The  hired  men  who  would  normally  unpack  them  were  not  working. 

3.  A  university  professor,  older  by  several  years  than  Herbert,  gave  a  look 
at  the  crates. 

4.  "Those  fellows  will  unpack  them  Monday  morning,"  he  said  lightly. 

5.  "Those  fellows.  Hell!  He'll  do  them  now,"  flared  Herbert. 

6.  Herbert  never  felt  inhibited  in  the  presence  of  men  older  or  higher  in  the 
academic  hierarchy. 

7.  Profanity  was  used  freely  at  the  Met  Lab. 

8.  The  professor  took  off  his  coat,  and  the  two  of  them  started  wrenching  at 

the  crates. 

9.  Would  Germany  get  atomic  weapons  before  the  United  States  developed  them? 

10.  Would  these  weapons  come  in  time  to  help  win  the  war? 

11.  These  unanswered  questions  were  constantly  in  the  minds  of  the  project 
leaders. 

12.  Their  presence  pressed  the  group  to  work  faster  and  faster,  to  be  tense, 
and  to  swear. 

13.  It  relieved  the  tension  built  up  by  having  to  work  against  time. 


Appendix  B 
Version  A- 19 

1.  Herbert  Anderson  and  his  group  at  the  lab  had  also  been  building  small 
piles. 

2.  The  best  place  they  had  been  able  to  find  for  work  on  the  pile  was  a  squash 
court  under  the  football  field. 

3.  The  physicists  would  have  liked  more  space,  but  the  expanding  armed  forces 
had  requisitioned  all  the  space. 

4.  They  were  still  very  "small  piles"  because  materials  came  in  at  a  very  slow 
if  steady  pace. 

5.  As  each  new  shipment  of  crates  arrived,  Herbert's  spirits  rose. 

6.  He  could  work  all  hours  and  drive  his  associates  to  work  along  with  his 
same  Intensity  and  enthusiasm. 

7.  A  shipment  of  crates  arrived  at  the  Chicago  Stadium  on  a  Saturday. 

8.  The  hired  men  who  would  normally  unpack  them  were  not  working. 

9.  A  university  professor,  older  by  several  years  than  Herbert,  gave  a  look  at 
the  crates. 

10.  "Those  fellows  will  unpack  them  Monday  morning,"  he  said  lightly. 

11.  "Those  fellows.  Hell!"  He'll  do  them  now,"  flared  Herbert. 

12.  Herbert  never  felt  Inhibited  In  the  presence  of  men  older  or  higher  in  the 
academic  hierarchy. 

13.  Profanity  was  freely  used  at  the  Met  Lab. 

14.  The  professor  took  off  his  coat,  and  the  two  of  them  started  wrenching  at 

the  crates. 

15.  Would  Germany  get  atomic  weapons  before  the  United  States  developed  them? 

16.  Would  these  weapons  come  In  time  to  help  win  the  war? 

17.  These  unanswered  questions  were  constantly  In  the  minds  of  the  project 
leaders. 


18.  Their  presence  pressed  the  group  to  work  faster  and  faster,  to  be  tense, 
and  to  swear. 

19.  It  relieved  the  tension  built  up  by  having  to  work  against  time. 


Appendix  C 
Version  B-13 
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1.  Herbert  Anderson  went  to  the  Goodyear  Tire  and  Rubber  Company  to  place  an 
order  for  a  square  balloon. 

2.  The  Goodyear  people  had  never  heard  of  square  balloons,  they  did  not  think 
they  could  fly. 

3.  At  first  they  threw  suspicious  glances  at  Herbert. 

4.  The  young  man,  however,  seemed  to  be  In  full  possession  of  his  wits. 

5.  He  talked  earnestly  and  knew  exactly  what  he  wanted. 

6.  The  Goodyear  people  promised  to  make  a  square  balloon  of  rubberized  cloth. 

7.  It  came  neatly  folded,  but  once  unfolded,  it  was  a  huge  thing  that  reached 
from  floor  to  ceiling. 

8.  The  room  ceiling  could  not  be  pushed  up  as  the  physicists  would  have  liked. 

9.  They  had  calculated  that  their  final  pile  ought  to  chain-react  somewhat 
before  It  reached  the  ceiling. 

10.  But  not  much  margin  was  left  and  calculations  are  never  to  be  trusted 
entirely. 

11.  In  addition,  some  unforeseen  factor  might  upset  theory. 

12.  The  critical  size  of  the  pile  might  not  be  reached  at  the  ceiling. 

13.  Since  the  room  Imposed  a  concrete  limit,  they  thought  of  improving  the 
performance  of  the  pile  other  than  by  size. 
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Appendix  D 
Version  B-19 

1.  Herbert  Anderson  went  to  the  Goodyear  Tire  and  Rubber  Company  to  place  an 
order  for  a  square  balloon. 

2.  The  Goodyear  people  had  never  heard  of  square  balloons,  they  did  not  think 
they  could  fly. 

3.  At  first  they  threw  suspicious  glances  at  Herbert. 

4.  The  young  man,  however,  seemed  to  be  in  full  possession  of  his  wits. 

5.  He  talked  earnestly  and  knew  exactly  what  he  wanted. 

6.  The  Goodyear  people  promised  to  make  a  square  balloon  of  rubberized  cloth. 

7.  They  delivered  it  a  couple  of  months  later. 

8.  It  came  neatly  folded,  but  once  unfolded,  it  was  a  huge  thing  that  reached 
from  floor  to  ceiling. 

9.  The  room  ceiling  could  not  be  pushed  up  as  the  physicists  would  have  liked. 

10.  They  had  calculated  that  their  final  pile  ought  to  chain-react  somewhat 
before  it  reached  the  ceiling. 

11.  But  not  much  margin  was  left  and  calculations  are  never  to  be  trusted 
entirely. 

12.  Some  impurities  In  the  pile  materials  might  go  unnoticed. 

13.  In  addition,  some  unforeseen  factor  might  upset  theory. 

14.  The  critical  size  of  the  pile  might  not  be  reached  at  the  ceiling. 

15.  Since  the  room  Imposed  a  concrete  limit,  they  thought  of  improving  the 
performance  of  the  pile  other  than  by  size. 

16.  There  had  been  an  experiment  at  Columbia  with  a  canned  pile. 

17.  The  Columbia  experiment  suggested  that  performance  could  be  Improved  by 
removing  air  from  the  graphite  pores. 

18.  To  can  as  large  a  pile  as  they  were  to  build  was  now  impractical. 


19.  They  could,  however,  assemble  their  pile  Inside  a  square  balloon  and  pump 
the  air  out. 


Appendix  E 

Recognition  Test:  A-13 
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1.  *Wou1d  they  be  ready  soon  enough  to  serve  as  weapons  In  winning  the  war? 

Would  these  weapons  come  In  time  to  help  win  the  war? 

2.  A  university  professor,  older  by  several  years  than  Herbert,  gave  a  look 
at  the  crates. 

♦A  university  professor  who  was  several  years  older  than  Herbert  looked  at 
the  crates. 

3.  A  shipment  of  crates  arrived  at  the  Chicago  stadium  on  a  Saturday. 

*It  was  a  Saturday  In  Chicago  when  the  shipment  of  crates  arrived. 

4.  Profanity  was  freely  used  at  the  Met  Lab. 

♦The  men  at  the  Met  Lab  used  profanity  freely. 

5.  These  unanswered  questions  were  constantly  in  the  minds  of  the  project 
1 eaders . 

*Leaders  of  the  project  had  these  unanswered  questions  constantly  In  their 
mi nds. 

6.  Herbert  never  felt  inhibited  in  the  presence  of  men  older  or  higher  in  the 
academic  hierarchy. 

♦Men  older  than  Herbert  or  higher  in  the  academic  hierarchy  never  Inhibited 
him. 

7.  *They  would  normally  be  unpacked  by  hired  men  who  were  not  working. 

The  hired  men  who  would  normally  unpack  them  were  not  working. 

8.  *In  their  presence,  the  group  was  pressed  to  work  faster  and  faster,  to  be 

tense,  and  to  swear. 

Their  presence  pressed  the  group  to  work  faster  and  faster,  to  be  tense, 
and  to  swear. 

9.  "Those  fellows  will  unpack  them  Monday  morning,"  he  said  lightly. 

♦"Monday  morning,"  he  said  lightly,  "those  fellows  will  be  here  to  unpack 
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them." 

10.  The  professor  took  off  his  coat,  and  the  two  of  them  started  wrenching  at 
the  crates. 

♦As  the  professor  took  his  coat  off,  the  two  of  them  started  wrenching  at 
the  crates. 

11.  "Those  fellows.  Hell!"  flared  Herbert,  "He  will  unpack  them  now." 

♦"Those  fellows.  Hell!  He'll  do  them  now,"  flared  Herbert. 

12.  It  relieved  the  tension  built  up  by  having  to  work  against  time. 

♦It  gave  relief  from  the  tension  of  having  to  work  against  time. 

13.  *Hould  atomic  weapons  be  developed  in  Germany  before  the  United  States  had 

them? 

Uould  Germany  get  atomic  weapons  before  the  United  States  developed  them? 
♦Paraphrased  Version 


Page  46 


Appendix  F 

Recognition  Test:  A-19 

1.  ♦"Those  fellows.  Hell!"  flared  Herbert,  "We  will  unpack  them  now." 

"Those  fellows.  Hell!  We'll  do  them  now,"  flared  Herbert. 

2.  *It  gave  relief  from  the  tension  of  having  to  work  against  time. 

It  relieved  the  tension  built  up  by  having  to  work  against  time. 

3.  *Since  materials  came  in  at  a  very  slow  though  steady  pace,  they  were  still 

very  “small  piles." 

They  were  still  very  "small  piles"  because  materials  came  in  at  a  very 
slow  if  steady  pace. 

4.  Would  Germany  get  atomic  weapons  before  the  United  States  developed  them? 
*Wou1d  atomic  weapons  be  developed  in  Germany  before  the  United  States  had 

them? 

5.  He  could  work  all  hours  and  drive  his  associates  to  work  along  with  his 
same  intensity  and  enthusiasm. 

♦He  worked  all  hours  and  drove  his  associates  to  work  along  with  his  same 
enthusiasm  and  intensity. 

6.  "Those  fellows  will  unpack  them  Monday  morning,"  he  said  lightly. 

♦“Monday  morning,"  he  said  lightly,  “those  fellows  will  be  here  to  unpack 

them." 

7.  *Would  they  be  ready  soon  enough  to  serve  as  weapons  in  winning  the  war? 

Would  these  weapons  come  in  time  to  help  win  the  war? 

8.  *Men  older  than  Herbert  or  higher  in  the  academic  hierarchy  never  inhibited 

him. 

Herbert  never  felt  inhibited  in  the  presence  of  men  older  or  higher  In  the 
academic  hierarchy. 


9.  A  shipment  of  crates  arrived  at  the  Chicago  stadium  on  a  Saturday 
♦It  was  Saturday  in  Chicago  when  the  shipment  of  crates  arrived. 


10.  *A  university  professor  who  was  several  years  older  than  Herbert  looked  at 

the  crates. 

A  university  professor,  older  by  several  years  than  Herbert,  gave  a  look 
at  the  crates. 

11.  Profanity  was  freely  used  at  the  Met  Lab. 

♦The  men  at  the  Met  Lab  used  profanity  freely. 

12.  *Leaders  of  the  project  has  these  unanswered  questions  constantly  in  their 

mi nds. 

These  unanswered  questions  were  constantly  in  the  minds  of  the  project 
leaders. 

13.  The  best  place  they  had  been  able  to  find  for  work  on  the  pile  was  a 
squash  court  under  the  football  field. 

♦A  squash  court  under  the  football  field  was  the  best  place  they  could  find 
for  work  on  the  pile. 

14.  The  hired  men  who  would  normally  unpack  them  were  not  working. 

♦They  would  normally  be  unpacked  by  hired  men  who  were  not  working. 

15.  *Although  the  physicists  would  have  liked  more  space.  It  had  all  been 

requisitioned  by  the  expanding  armed  forces. 

The  physicists  would  have  liked  more  space,  but  the  expanding  armed  forces 
had  requisitioned  all  the  space. 

16.  *As  the  professor  took  his  coat  off,  the  two  of  them  started  wrenching  at 

the  crates. 

The  professor  took  off  his  coat,  and  the  two  of  them  started  wrenching  at 
the  crates. 

17.  As  each  new  shipment  of  crates  arrived,  Herbert's  spirits  rose. 

♦Herbert's  spirits  rose  with  the  arrival  of  each  new  shipment  of  crates. 
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18.  Their  presence  pressed  the  group  to  work  faster  and  faster,  to  be  tense, 
and  to  swear. 

♦In  their  presence  the  group  was  pressed  to  work  faster  and  faster,  to  be 
tense,  and  to  swear. 

19.  ^Herbert  Anderson  had  also  been  building  small  piles  with  his  group  at  the 

lab. 

Herbert  Anderson  and  his  group  at  the  lab  had  also  been  building  small 
piles. 

♦Paraphrased  Version 


Appendix  G 

Recognition  Test:  B-13 

1.  The  people  at  Goodyear  promised  to  make  his  square  balloon  using 
rubberized  cloth. 

*The  Goodyear  people  promised  to  make  a  square  balloon  of  rubberized 
cloth. 

2.  But  not  much  margin  was  left  and  calculations  are  never  to  be  trusted 
entirely. 

♦Not  much  margin  was  left,  however,  and  calculations  are  never  entirely 
to  be  trusted. 

3.  Herbert  Anderson  went  to  the  Goodyear  Tire  and  Rubber  Company  to  place 
an  order  for  a  square  balloon. 

♦A  square  balloon  was  ordered  by  Herbert  Anderson  at  the  Goodyear  Tire 
and  Rubber  Company. 

4.  *The  pile  might  not  reach  critical  size  at  the  ceiling. 

The  critical  size  of  the  pile  might  not  be  reached  at  the  ceiling. 

5.  The  young  man,  however,  seemed  to  be  in  full  possession  of  his  wits. 
♦However,  the  young  man  seemed  fully  in  possession  of  his  wits. 

6.  Since  the  room  Imposed  a  concrete  limit,  they  thought  of  improving  the 
performance  of  the  pile  other  than  by  size. 

♦They  thought  of  ways  of  improving  performance  of  the  pile  other  than  by 
size,  because  the  room  Imposed  a  concrete  limit. 

7.  In  addition,  some  unforeseen  factor  might  upset  theory. 

♦Further,  theory  might  be  upset  by  some  unforeseen  factor. 

8.  *He  spoke  earnestly,  knowing  exactly  what  he  wanted. 

He  talked  earnestly  and  knew  exactly  what  he  wanted. 
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9.  *By  their  calculations,  the  final  pile  should  chain- react  a  bit  before 
reaching  the  celling. 

They  had  calculated  that  their  final  pile  ought  to  chaln-react  somewhat 
before  It  reached  the  celling. 

10.  At  first  they  threw  suspicious  glances  at  Herbert. 

♦They  gave  Herbert  suspicious  glances  at  first. 

11.  The  room  ceiling  could  not  be  pushed  up  as  the  physicists  would  have 
liked. 

♦The  ceiling  of  the  room  could  not  be  raised  as  these  physicists  would 
have  wished. 

12.  *It  came  folded  neatly,  but  it  was  a  huge  thing  and  once  unfolded  It 

reached  from  floor  to  ceiling. 

It  came  neatly  folded,  but  once  unfolded,  it  was  a  huge  thing  that 
reached  from  floor  to  celling. 

13.  *They  had  never  heard  of  square  balloons  at  Goodyear  and  they  did  not  think 

they  could  fly. 

The  Goodyear  people  had  never  heard  of  square  balloons,  they  did  not  think 
they  could  fly. 

♦Paraphrased  Version 
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Appendix  H 

Recognition  Test:  B-19 

1.  He  talked  earnestly  and  knew  exactly  what  he  wanted. 

♦He  spoke  earnestly,  knowing  exactly  what  he  wanted. 

2.  *From  the  Columbia  experiment  It  appeared  that  performance  might  be 

improved  by  removing  air  from  the  pores  In  the  graphite. 

The  Columbia  experiment  suggested  that  performance  could  be  Improved  by 
removing  air  from  the  graphite  pores. 

3.  *It  came  folded  neatly,  but  It  was  a  huge  thing  and  once  unfolded  It 

reached  from  floor  to  celling. 

It  came  neatly  folded,  but  once  unfolded,  it  was  a  huge  thing  that  reached 
from  floor  to  ceiling. 

4.  *The  celling  of  the  room  could  not  be  raised  as  these  physicists  would  have 

wished. 

The  room  celling  could  not  be  pushed  up  as  the  physicists  would  have 
liked. 

5.  ^Materials  In  the  pile  might  have  unnoticed  Impurities. 

Some  Impurities  in  the  pile  materials  might  go  unnoticed. 

6.  At  first  they  threw  suspicious  glances  at  Herbert. 

♦They  gave  Herbert  suspicious  glances  at  first. 

7.  *The  Goodyear  people  promised  to  make  a  square  balloon  of  rubberized  cloth. 

The  people  at  Goodyear  promised  to  make  his  square  balloon  using 
rubberized  cloth. 

8.  ^However,  the  young  man  seemed  fully  In  possession  of  his  wits. 

The  young  man,  however,  seemed  to  be  in  full  possession  of  his  wits. 

i 
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9.  *By  their  calculations,  the  final  pile  should  chaln-react  a  bit  before 
reaching  the  celling. 

They  had  calculated  that  their  final  pile  ought  to  chaln-react  somewhat 
before  It  reached  the  celling. 

10.  Herbert  Anderson  went  to  the  Goodyear  Tire  and  Rubber  Company  to  place  an 
order  for  a  square  balloon. 

*A  square  balloon  was  ordered  by  Herbert  Anderson  at  the  goodyear  Tire  and 
Rubber  Company. 

11.  ♦Further,  theory  might  be  upset  by  some  unforeseen  factor. 

In  addition,  some  unforeseen  factor  might  upset  theory. 

12.  They  delivered  it  a  couple  of  months  later. 

♦It  was  delivered  a  couple  of  months  later. 

13.  To  can  as  large  a  pile  as  they  were  to  build  was  now  impractical. 

*It  was  now  Impractical  to  can  a  pile  as  large  as  they  were  building. 

14.  *At  Columbia  there  had  been  an  experiment  with  a  canned  pile. 

There  had  been  an  experiment  at  Columbia  with  a  canned  pile. 

15.  *They  had  never  heard  of  square  balloons  at  Goodyear  and  they  did  not  think 

they  could  fly. 

The  Goodyear  people  had  never  heard  of  square  balloons,  they  did  not  think 
they  could  fly. 

16.  The  critical  size  of  the  pile  might  not  be  reached  at  the  celling. 

♦The  pile  might  not  reach  critical  size  at  the  celling. 

17.  *Not  much  margin  was  left,  however,  and  calculations  are  never  entirely 

to  be  trusted. 

But  not  much  margin  was  left  and  calculations  are  never  to  be  trusted 
entirely. 


18.  *The  pile  could  be  assembled  Inside  the  square  balloon,  however,  and  the 

air  could  be  pumped  out. 

The  could,  however,  assemble  their  pile  Inside  a  square  balloon  and  pump 
the  air  out. 

19.  Since  the  room  Imposed  a  concrete  limit,  they  thought  of  Improving  the 
performance  of  the  pile  other  than  by  size. 

♦They  thought  of  ways  of  Improving  performance  of  the  pile  other  than  by 
size,  because  the  room  Imposed  a  concrete  limit. 


♦Paraphrased  Version 


